The auditory and the visual percept

In analyses and models of audiovisual speech perception, it has been common to consider three percepts: (1) the auditory percept evoked by acoustic stimuli, (2) the visual percept evoked by optic stimuli and (3) a common percept evoked by synchronous optic and acoustic stimuli. Here, it is shown that a vocal percept that is heard and influenced by vision has to be distinguished from a gestural percept that is seen and influenced by audition. In the two experiments reported, syllables distinguished solely by their vowels [i], [y] or [e] were presented to phonetically sophisticated subjects auditorily, visually and in incongruently cross-dubbed audiovisual form. In the first, the subjects rated roundedness, lip spreading, openness and backness of the vowels they heard in the second of those they saw. The results confirmed that roundedness is mainly heard by eye while openness is heard by ear. Heard backness (retraction) varied with the acoustic and optic presence of roundedness. Seen openness was substantially influenced by acoustic cues, while there was no such influence on seen roundedness. The results are discussed in the context of theories and models.